Scanning system and calibration thereof

ABSTRACT

Calibrating an intraoral scanner includes obtaining reference data of a reference three-dimensional (3D) representation of a calibration object and obtaining, based on the intraoral scanner being used by a user to scan the 3D calibration object, and from one or more device to real-world coordinate transformations of two-dimensional (2D) images of the 3D calibration object, measurement data. Calibrating the intraoral scanner further includes aligning the measurement data to the reference data to obtain alignment data and updating, based on the alignment data, said one or more transformations.

RELATED APPLICATIONS

This patent application is a continuation application of U.S. patentapplication Ser. No. 17/740,113 filed May 9, 2022, which is acontinuation application of U.S. patent application Ser. No. 17/492,453filed Oct. 1, 2021, which is a continuation application of U.S. patentapplication Ser. No. 17/161,503 filed Jan. 28, 2021, which is acontinuation application of U.S. patent application Ser. No. 16/823,156filed Mar. 18, 2020, which is a continuation application of U.S. patentapplication Ser. No. 16/708,293 filed Dec. 9, 2019, which is acontinuation application of U.S. patent application Ser. No. 16/396,484filed Apr. 26, 2019, which is a continuation of U.S. patent applicationSer. No. 16/286,437, filed Feb. 26, 2019, which is a continuationapplication of U.S. patent application Ser. No. 15/610,515, filed May31, 2017, which is a divisional application of U.S. patent applicationSer. No. 14/825,173, filed Aug. 13, 2015, which claims the benefit under35 U.S.C. § 119(e) of U.S. Provisional Application No. 62/037,778, filedAug. 15, 2014, all of which are herein incorporated by reference.

TECHNICAL FIELD

Embodiments of the present invention relate to the field of imaging and,in particular, to a system and method for performing confocal imaging ofa three dimensional surface.

BACKGROUND

A great variety of methods and systems have been developed for directoptical measurement of teeth and the subsequent automatic manufacture ofdentures. The term “direct optical measurement” signifies surveying ofteeth in the oral cavity of a patient. This facilitates the obtainmentof digital constructional data necessary for the computer-assisteddesign (CAD) or computer-assisted manufacture (CAM) of toothreplacements without having to make any cast impressions of the teeth.Such systems typically include an optical probe coupled to an opticalpick-up or receiver such as charge coupled device (CCD) or complementarymetal-oxide semiconductor (CMOS) sensor and a processor implementing asuitable image processing technique to design and fabricate virtuallythe desired product.

One type of system that performs intra-oral scans is a system that usesconfocal imaging to image a three dimensional surface. Such systems thatuse confocal imaging typically include field lenses to flatten animaging field and enable flat focal planes for emitted light beams. Suchflat focal planes ensure that the surface topology of scanned threedimensional surfaces is accurate. However, the field lenses arediverging lenses that open the rays of the light beams. This causes theoptics of the confocal imaging apparatus to be enlarged. Additionally,the field lenses should be aligned to ensure accuracy. Such alignmentcan be a time consuming and challenging process.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention is illustrated by way of example, and not by wayof limitation, in the figures of the accompanying drawings.

FIG. 1A illustrates a functional block diagram of a confocal imagingapparatus according to one embodiment.

FIG. 1B illustrates a block diagram of a computing device that connectsto a confocal imaging apparatus, in accordance with one embodiment.

FIG. 2A illustrates optics of a confocal imaging apparatus that lacks afield lens, in accordance with one embodiment.

FIG. 28 illustrates optics of a confocal imaging apparatus that lacks afield lens, in accordance with another embodiment.

FIG. 2C illustrates optics of a confocal imaging apparatus with afieldlens for which changes in a focusing setting cause changes inmagnification, in accordance with another embodiment.

FIG. 3A is a top view of a probing member of a confocal imagingapparatus that includes a prism, in accordance with an embodiment of theinvention.

FIG. 38 is a longitudinal cross-section through line II-II of theprobing member in FIG. 3A.

FIG. 3C is a view of a probing member that includes an internal target,in accordance with one embodiment.

FIG. 3D is a side view of a probing member that includes an internaltarget, in accordance with one embodiment.

FIG. 4 is a schematic illustration of optics of a confocal imagingapparatus, in accordance with one embodiment.

FIG. 5A is a flowchart showing one embodiment of a method forcalibrating a confocal imaging apparatus having an imaginary non-flatfocal surface.

FIG. 5B is a flowchart showing one embodiment of a method forcalibrating a confocal imaging apparatus for which changes in a focusingsetting cause changes in magnification.

FIG. 5C illustrates one example calibration object, in accordance withone embodiment.

FIG. 5D illustrates a chart showing a distribution of points of acalibration object as measured by a confocal imaging apparatus, inaccordance with one embodiment.

FIG. 5E illustrates a chart showing a distribution of points in a worldcoordinate system, in accordance with one embodiment.

FIG. 6 is a flowchart showing one embodiment of a method for adjustingdepth measurements of a scanned three dimensional object based onapplication of a field curvature model calibrated to a confocal imagingapparatus.

FIG. 7 illustrates a block diagram of an example computing device, inaccordance with embodiments of the present invention.

DETAILED DESCRIPTION

Described herein is a confocal imaging apparatus having a non-flat focalsurface. The non-flat focal surface may be caused by the optics of theconfocal imaging apparatus lacking a field lens. As is discussed ingreater detail below, the lack of a field lens in the confocal imagingapparatus introduces challenges but also provides numerous advantages.For example, a confocal imaging apparatus without a field lens issmaller, lighter and easier to manufacture than a confocal imagingapparatus having a field lens. Embodiments discussed herein show how toovercome the challenges in designing and using a confocal imagingapparatus lacking a field lens.

Also described herein is a large field confocal imaging apparatus havingfocusing optics that change a magnification of a focal surface withchanges in a focusing setting. As is discussed in greater detail below,the change in magnification introduces challenges that are overcome inembodiments.

In one embodiment, a confocal imaging apparatus includes an illuminationmodule to generate an array of light beams. Focusing optics of theconfocal imaging apparatus perform confocal focusing of an array oflight beams onto a non-flat focal surface and direct the array of lightbeams toward a three dimensional object to be imaged. A translationmechanism of the confocal imaging apparatus adjusts a location of atleast one lens to displace the non-flat focal surface along an imagingaxis. A detector of the confocal imaging apparatus measures intensitiesof an array of returning light beams that are reflected off of the threedimensional object and directed back through the focusing optics.Intensities of the array of returning light beams are measured forlocations of the at least one lens for determination of positions on theimaging axis of points of the three dimensional object. Detectedpositions of one or more points are adjusted to compensate for thenon-flat focal surface. Thus, an object may be accurately imaged despitethe non-flat focal surface of the confocal imaging apparatus.

FIG. 1A illustrates a functional block diagram of a confocal imagingapparatus 20 according to one embodiment. FIG. 1B illustrates a blockdiagram of a computing device 24 that connects to the confocal imagingapparatus 20. Together, the confocal imaging apparatus 20 and computingdevice 24 may form a system for generating three dimensional images ofscanned objects. The computing device 24 may be connected to theconfocal imaging apparatus 20 directly or indirectly and via a wired orwireless connection. For example, the confocal imaging apparatus 20 mayinclude a network interface controller (NIC) capable of communicatingvia Wi-Fi, via third generation (3G) or fourth generation (4G)telecommunications protocols (e.g., global system for mobilecommunications (GSM), long term evolution (LTE), Wi-Max, code divisionmultiple access (CDMA), etc.), via Bluetooth, via Zigbee, or via otherwireless protocols. Alternatively, or additionally, confocal imagingapparatus may include an Ethernet network interface controller (NIC), auniversal serial bus (USB) port, or other wired port. The NIC or portmay connect the confocal imaging apparatus to the computing device via alocal area network (LAN). Alternatively, the confocal imaging apparatus20 may connect to a wide area network (WAN) such as the Internet, andmay connect to the computing device 24 via the WAN. In an alternativeembodiment, confocal imaging apparatus 20 is connected directly to thecomputing device (e.g., via a direct wired or wireless connection). Inone embodiment, the computing device 24 is a component of the confocalimaging apparatus 20.

Referring now to FIG. 1A, in one embodiment confocal imaging apparatus20 includes a semiconductor laser unit 28 that emits a focused lightbeam, as represented by arrow 30. The light beam 30 passes through apolarizer 32. Polarizer 32 polarizes the light beam passing throughpolarizer 32. Alternatively, polarizer 32 may be omitted in someembodiments. The light beam then enters into an optic expander 34 thatimproves a numerical aperture of the light beam 30. The light beam 30then passes through an illumination module 38, which splits the lightbeam 30 into an array of incident light beams 36, represented here, forease of illustration, by a single line. The illumination module 38 maybe, for example, a grating or a micro lens array that splits the lightbeam 30 into an array of light beams 36. In one embodiment, the array oflight beams 36 is an array of telecentric light beams. Alternatively,the array of light beams may not be telecentric.

The confocal imaging apparatus 20 further includes a unidirectionalmirror or beam splitter (e.g., a polarizing beam splitter) 40 thatpasses the array of light beams 36. A unidirectional mirror 40 allowstransfer of light from the semiconductor laser 28 through to downstreamoptics, but reflects light travelling in the opposite direction. Apolarizing beam splitter allows transfer of light beams having aparticular polarization and reflects light beams having a different(e.g., opposite) polarization. In one embodiment, the unidirectionalmirror or beam splitter 40 has a small central aperture. The smallcentral aperture may improve a measurement accuracy of the confocalimaging apparatus 20. In one embodiment, as a result of a structure ofthe unidirectional mirror or beam splitter 40, the array of light beamswill yield a light annulus on an illuminated area of an imaged object aslong as the area is not in focus. Moreover, the annulus will become acompletely illuminated spot once in focus. This ensures that adifference between measured intensities of out-of focus points andin-focus points will be larger.

Along an optical path of the array of light beams after theunidirectional mirror or beam splitter 40 are confocal focusing optics42, and an endoscopic probing member 46. Additionally, a quarter waveplate may be disposed along the optical path after the unidirectionalmirror or beam splitter 40 to introduce a certain polarization to thearray of light beams. In some embodiments this may ensure that reflectedlight beams will not be passed through the unidirectional mirror or beamsplitter 40. Confocal focusing optics 42 may additionally include relayoptics (not shown). Confocal focusing optics 42 may or may not maintainthe same magnification of an image over a wide range of distances in theZ direction, wherein the Z direction is a direction of beam propagation(e.g., the Z direction corresponds to an imaging axis that is alignedwith an optical path of the array of light beams 36). The relay opticsenable the confocal imaging apparatus 20 to maintain a certain numericalaperture for propagation of the array of light beams 36. The confocalfocusing optics 42 and endoscopic probing member 46 are discussed ingreater detail with reference to FIGS. 2A-2C.

The endoscopic probing member 46 may include a rigid, light-transmittingmedium, which may be a hollow object defining within it alighttransmission path or an object made of alight transmitting material,e.g. a glass body or tube. In one embodiment, the endoscopic probingmember 46 include a prism such as a folding prism. At its end, theendoscopic probing member 46 may include a mirror of the kind ensuring atotal internal reflection. Thus, the mirror may direct the array oflight beams towards a teeth segment 26 or other object. The endoscopeprobing member 46 thus emits array of light beams 48, which impinge onto surfaces of the teeth section 26.

The array of light beams 48 are arranged in an X-Y plane, in theCartesian frame 50, propagating along the Z axis. As the surface onwhich the incident light beams hits is an uneven surface, illuminatedspots 52 are displaced from one another along the Z axis, at different(X_(i), Y_(i)) locations. Thus, while a spot at one location may be infocus of the confocal focusing optics 42, spots at other locations maybe out-of-focus. Therefore, the light intensity of returned light beamsof the focused spots will be at its peak, while the light intensity atother spots will be off peak. Thus, for each illuminated spot, multiplemeasurements of light intensity are made at different positions alongthe Z-axis. For each of such (X_(i), Y_(i)) location, the derivative ofthe intensity over distance (Z) may be made, with the Z_(i) yieldingmaximum derivative, Z₀, being the in-focus distance. As pointed outabove, the incident light from the array of light beams 48 forms a lightdisk on the surface when out of focus and a complete light spot when infocus. Thus, the distance derivative will be larger when approachingin-focus position, increasing accuracy of the measurement.

The light scattered from each of the light spots includes a beamtravelling initially in the Z axis along the opposite direction of theoptical path traveled by the array of light beams 48. Each returnedlight beam in an array of returning light beams 54 corresponds to one ofthe incident light beams in array of light beams 36. Given theasymmetrical properties of unidirectional mirror or beam splitter 40,the returned light beams are reflected in the direction of detectionoptics 60.

The detection optics 60 may include a polarizer 62 that has a plane ofpreferred polarization oriented normal to the plane polarization ofpolarizer 32. Alternatively, polarizer 32 and polarizer 62 may beomitted in some embodiments. The array of returning light beams 54 maypass through imaging optics 64 in one embodiment. The imaging optics 64may be one or more lenses. Alternatively, the detection optics 60 maynot include imaging optics 64. In one embodiment, the array of returninglight beams 54 further passes through a matrix 66, which may be an arrayof pinholes. Alternatively, no matrix 66 is used in some embodiments.The array of returning light beams 54 are then directed onto a detector68.

The detector 68 is an image sensor having a matrix of sensing elementseach representing a pixel of the image. If matrix 66 is used, then eachpixel further corresponds to one pinhole of matrix 66. In oneembodiment, the detector is a charge coupled device (CCD) sensor. In oneembodiment, the detector is a complementary metal-oxide semiconductor(CMOS) type image sensor. Other types of image sensors may also be usedfor detector 68. The detector 68 detects light intensity at each pixel.

In one embodiment, detector 68 provides data to computing device 24.Thus, each light intensity measured in each of the sensing elements ofthe detector 68, is then captured and analyzed, in a manner to bedescribed below, by processor 24.

Confocal imaging apparatus 20 further includes a control module 70connected both to semiconductor laser 28 and a motor 72, voice coil orother translation mechanism. In one embodiment, control module 70 is orincludes a field programmable gate array (FPGA) configured to performcontrol operations. Motor 72 is linked to confocal focusing optics 42for changing a focusing setting of confocal focusing optics 42. This mayadjust the relative location of an imaginary non-flat focal surface ofconfocal focusing optics 42 along the Z-axis (e.g., in the imagingaxis). Control module 70 may induce motor 72 to axially displace (changea location of) one or more lenses of the confocal focusing optics 42 tochange the focal depth of the imaginary non-flat focal surface. In oneembodiment, motor 72 or confocal imaging apparatus 20 includes anencoder (not shown) that accurately measures a position of one or morelenses of the confocal focusing optics 42. The encoder may include asensor paired to a scale that encodes a linear position. The encoder mayoutput a linear position of the one or more lenses of the confocalfocusing optics 42. The encoder may be an optical encoder, a magneticencoder, an inductive encoder, a capacitive encoder, an eddy currentencoder, and so on. After receipt of feedback that the location of theone or more lenses has changed, control module 70 may induce laser 28 togenerate a light pulse. Control unit 70 may additionally synchronizeimage-capturing module 80 from FIG. 1B to receive and/or store datarepresentative of the light intensity from each of the sensing elementsat the particular location of the one or more lenses (and thus of thefocal depth of the imaginary non-flat focal surface). In subsequentsequences, the location of the one or more lenses (and thus the focaldepth) will change in the same manner and the data capturing willcontinue over a wide focal range of confocal focusing optics 42.

Referring now to FIG. 1B, image capturing module 80 may capture imagesresponsive to receiving image capture commands from the control unit 70.The captured images may be associated with a particular focusing setting(e.g., a particular location of one or more lenses in the confocalfocusing optics as output by the encoder). Image processing module 82then processes captured images captured over multiple different focusingsettings. Image processing module 82 includes a depth determiner 90 anda field compensator 92 for processing image data.

Depth determiner 90 determines the relative intensity in each pixel overthe entire range of focal settings of confocal focusing optics 42 fromreceived image data. Once a certain light spot associated with aparticular pixel is in focus, the measured intensity will be maximal forthat pixel. Thus, by determining the Z_(i) corresponding to the maximallight intensity or by determining the maximum displacement derivative ofthe light intensity, for each pixel, the relative position of each lightspot along the Z axis can be determined for each pixel. Thus, datarepresentative of the three-dimensional pattern of a surface in theteeth segment 26 or other three dimensional object can be obtained.

In embodiments, the confocal focusing optics 42 of confocal imagingapparatus 20 lack field lenses. The purpose of the field lens is toflatten a focal field and thus produce a flat focal pane for the arrayof light beams. For confocal imaging apparatuses with field lenses, eachlight beam from the array of light beams focuses on the same flat focalpane. However, without such field lenses the array of light beams focuson an imaginary non-flat focal surface (e.g., on a curved focalsurface). This causes the Z axis information that depth determiner 90computes to be distorted for many pixels.

Field compensator 92 compensates for the curved field caused by the lackof a field lens. Field compensator 92 may also compensate for changes ina position of the curved focal surface caused by temperature and/or formagnification changes caused by changes in a focusing setting. Fieldcompensator 92 applies a field curvature model 94 and/or other opticscompensation model (not shown) to each Z axis measurement of each pixelto correct for field curvature, temperature and/or magnificationchanges. In one embodiment, a different field curvature model 94 (orother optics compensation model) is applied for each focusing setting ofthe confocal imaging apparatus 20. This is because the amount of fieldcurvature and/or magnification may change with changes in the focusingsetting. Alternatively, a single field curvature model 94 (or otheroptics compensation model) may account for the changes in the fieldcurvature caused by changes in the focusing setting and/or for changesin magnification caused by changes in the focusing setting. For eachcombination of an X,Y pixel location and a focusing setting (e.g., az-axis position of one or more lenses of the focusing optics), aparticular depth adjustment may be applied based on the field curvaturemodel or models. Additionally, an X location adjustment and/or a Ylocation adjustment bay be applied based on the field curvature modeland/or other optics compensation model. In one embodiment, for eachcombination of an X,Y pixel location, a focusing setting, and atemperature reading or a z-axis position of a measured element whoseposition changes with changes in temperature, a particular depthadjustment may be applied based on the field curvature model or models.The adjusted depth (z-axis) values represent the actual z-axis values ofthe imaged surface.

A three-dimensional representation may be constructed based on thecorrected measurement data and displayed via a user interface 84. Theuser interface 84 may be a graphical user interface that includescontrols for manipulating a display of the three-dimensionalrepresentation (e.g., viewing from different angles, zooming-in or out,etc.). In addition, data representative of the surface topology of thescanned object may be transmitted to remote devices by a communicationmodule 88 for further processing or use (e.g., to generate a threedimensional virtual model of the scanned object).

By capturing, in this manner, an image from two or more angularlocations around the structure, e.g. in the case of a teeth segment fromthe buccal direction, from the lingual direction and optionally fromabove the teeth, an accurate three-dimensional representation of theteeth segment may be reconstructed. This may allow a virtualreconstruction of the three-dimensional structure in a computerizedenvironment or a physical reconstruction in a CAD/CAM apparatus. Forexample, a particular application is imaging of a segment of teethhaving at least one missing tooth or a portion of a tooth. In such aninstance, the image can then be used for the design and subsequentmanufacture of a crown or any other prosthesis to be fitted into thisteeth segment.

FIG. 2A illustrates optics 200 of a confocal imaging apparatus thatlacks a field lens, in accordance with one embodiment. The optics 200may correspond to optics of confocal imaging apparatus 20 of FIG. 1A,such as confocal focusing optics 42.

The optics 200 include an illumination module 38, a unidirectionalmirror or beam splitter 40, a series of lenses that may correspond toconfocal focusing optics 42, and folding prism 220 arranged along anoptical path traversed by an array of light beams 225. The optical pathis shown to be a linear path. However, in embodiments one or more of thecomponents of optics 200 may change a direction of the optical path. Forexample, the folding prism 220 may include a mirror (not shown) that mayreflect light beams at an angle. An example of such a folding prism isshown in FIG. 3B. Referring back to FIG. 2 , an imaging axis 240 isshown that is aligned to the optical path traversed by the array oflight beams 225. The imaging axis 240 is a Z-axis that represents depth.As used herein, the imaging axis (or Z axis) may be a curvilinearcoordinate axis that corresponds to the optical path. Thus, if theoptical path changes direction, the imaging axis changes directioncorrespondingly.

Illumination module 38 is a source of multiple light beams. In oneembodiment, illumination module is a micro lens array that divides anincoming light beam into array of light beams 225. In one embodiment,the array of light beams output by the illumination module 38 is anarray of telecentric light beams. Accordingly, chief rays of the arrayof light beams may be parallel to each other. Unidirectional mirror orbeam splitter 40 is disposed along the optical path of the array oflight beams, and passes the array of light beams received from theunidirectional mirror or beam splitter 40.

In one embodiment, the confocal focusing optics are divided into aseries of lens groups including a first lens group 205, a second lensgroup 215 and a third lens group 210. First and/or second lens groups205, 215 may act as relay optics. The first and second lens groups 205,215 are configured to focus the array of light beams and compensate foroptical aberrations. Optical aberrations that may be corrected includeshape aberrations, coma, stigmatism, and so forth. In one embodiment,the first and second lens groups 205, 215 are configured to produce anapproximately rectangular field having minimal optical distortion. Thefirst lens group 205 and second lens group 215 may have a fixed positionrelative to each other and to other components of the optics 200. Thethird lens group 210 has a variable location that may be adjusted tochange a location of a curved focal surface produced by the optics 200.

The third lens group 210 is movable along the imaging axis (z axis), buthas a fixed position normal to the imaging axis. A focusing setting ofthe focusing optics can be adjusted by moving the third lens group 210along the imaging axis. Third lens group 210 may be adjusted to performscanning of an object. To scan an object, the third lens group 210 maybe displaced to numerous different locations (encoder positions) alongthe imaging axis 240, and images may be taken at each location. In oneembodiment, an axial gain of the focusing optics is approximately 7×.Accordingly, a displacement of the third lens group 210 adjusts alocation of a curved focal surface 230 by seven times the amount ofdisplacement. For example, a 1 mm displacement of the third lens group210 causes a position of the curved focal surface (also referred to as acurved focal plane) by 7 mm. This enables the optics 200 to be compactand minimizes movement during operation.

In one embodiment, second lens group 215 focuses the array of lightbeams 225 into prism 220, which may be a folding prism. Prism 220 may beconfigured to provide an appropriate refractive index (e.g., thatcorresponds to a refractive index of glass).

The optics 200 lack any field lens. A field lens is used to flatten afocal surface (flatten an imaging field) to achieve a flat focal plane.As shown, there is no field lens between the illumination module 38 andthe unidirectional mirror or beam splitter 40. Nor is there a field lensnear prism 220 or a field lens between the unidirectional mirror or beamsplitter 40 and a detector (not shown). The lack of a field lensintroduces numerous advantages over confocal imaging apparatuses thatuse field lenses. The field lens is a diverging lens that causes aradius of the lenses used for the focusing optics and/or for relayoptics to be larger. This in turn increases the amount of material(e.g., glass) used in the lenses and thus increases a weight of theconfocal imaging apparatus. Additionally, the larger lenses cause athickness of the confocal imaging apparatus to be larger. For example,an example confocal imaging apparatus with a field lens includes alargest lens having a distance from an optical axis to an outerperimeter of the lens of about 15 mm. In contrast, the same confocalimaging apparatus without a field lens may include a largest lens havinga distance from the optical axis to an outer perimeter of the lens ofless than 15 mm (e.g., less than 13 mm or about 9 mm in embodiments).

In a confocal imaging apparatus having a field lens, the field lens maybe positioned between the illumination module 38 and the unidirectionalmirror or beam splitter 40. This causes a spacing between theillumination module 38 and the unidirectional mirror or beam splitter 40to be about 7 mm. Additionally, a corresponding field lens would beplaced between the unidirectional mirror or beam splitter 40 and adetector (not shown) at a distance of about 7 mm. In contrast, byeliminating the field lens, the distance 235 between the illuminationmodule 38 and the unidirectional mirror or beam splitter 40 may be lessthan 7 mm (e.g., less than 5 mm or about 2 mm in embodiments). Thisfurther reduces the size of the confocal imaging apparatus.

As mentioned, if a field lens is used in a confocal imaging apparatus,then in actuality two field lenses are used. These two field lensesshould be matching field lenses and should be carefully aligned to oneanother. This alignment can be a time consuming process. Additionally,failure to exactly align these field lenses introduces inaccuracy intothe confocal imaging apparatus. Accordingly, an accuracy of the confocalimaging apparatus can be improved and an ease of manufacture for theconfocal imaging apparatus can be improved by eliminating the fieldlens.

The lack of a field lens causes the focal surface 230 to be a curvedfocal surface (or other non-flat focal surface). The shape of the curvedfocal surface 230 may depend on the focusing setting of the focusingoptics (e.g., the location of the third lens group 210). The curvedfocal surface may introduce significant error into the confocal imagingapparatus, which accounts for the inclusion of field lenses in priorconfocal imaging apparatuses. However, embodiments of the presentinvention provide a field compensator (see, e.g., field compensator 92of FIG. 1B) that minimizes or eliminates the error introduced by thelack of a field lens.

As shown, the confocal focusing optics is a non-telecentric opticalsystem. Accordingly, magnification of an imaged object may change withchanges in depth and/or in changes of focal settings. However, suchmagnification changes (and any accompanying distortion) may beaccommodated and corrected by the field compensator based on applicationof a field curvature model. Alternatively, the confocal focusing opticsmay operate in a telecentric mode, and distance-introduced magnificationchanges may be avoided.

FIG. 28 illustrates optics 250 of a confocal imaging apparatus thatlacks a field lens, in accordance with one embodiment. The optics 250may correspond to optics of confocal imaging apparatus 20 of FIG. 1A,such as confocal focusing optics 42. Similar to optics 200, optics 250include an illumination module 38, a unidirectional mirror (or beamsplitter) 40, and a series of lens groups. The series of lens groupsinclude a first lens group 255 with a fixed position and a second lensgroup 265 that is movable along an imaging axis 280 corresponding to adirection of propagation for an array of light beams 270.

The array of light beams 270 are focused onto a curved focal surface275. Though the optics 250 are not telecentric, magnification ispreserved (fixed) with changes in focusing settings because the array oflight beams are collimated between first lens group 255 and second lensgroup 265. For optics 250, axial gain is 1×. Accordingly, a displacementof 1 mm of the second lens group 265 causes a displacement of the curvedfocal surface of 1 mm.

An object may be placed along the beam path to be imaged. The array oflight beams 285 reflect off of the object and an array of returninglight beams return back through the series of lens groups. The array ofreturning light beams 285 is then reflected by the unidirectional mirror(or beam splitter) 40 onto detector 68. As shown, the optics 250 lack afield lens between the unidirectional mirror or beam splitter 40 and theillumination module 38 and further lack a field lens between theunidirectional mirror or beam splitter 40 and the detector 68.Accordingly, the focal surface for the optics 250 is a curved focalsurface 275.

Embodiments have been discussed herein with reference to a confocalimaging apparatus that lacks a field lens and that has a curved focalsurface. However, in some embodiments the confocal imaging apparatusincludes one or more field lenses and thus has a flat focal surface. Forsuch embodiments, the confocal imaging apparatus operates in anon-telecentric mode, and magnification at a focal plane changes withchanges in focusing settings of the confocal imaging apparatus.

FIG. 2C illustrates one example of optics 285 for a confocal imagingapparatus that includes a field lens, in accordance with one embodiment.The optics 285 may correspond to optics of confocal imaging apparatus 20of FIG. 1A, such as confocal focusing optics 42. Similar to optics 200and optics 250, optics 285 include an illumination module 38, aunidirectional mirror (or beam splitter) 40, and a series of lensgroups. However, optics 285 also include a field lens 288 that causes aflat focal plane 299. The series of lens groups include a first lensgroup 290 with a fixed position, a second lens group 292 with a fixedposition and a third lens group 294 that is movable along an imagingaxis 297 corresponding to a direction of propagation for an array oflight beams 298.

The array of light beams 298 are focused onto flat focal plane 299.Magnification at the flat foal plane 299 changes with changes infocusing settings. The changes in magnification may introducesignificant error into the confocal imaging apparatus. Accordingly, thefocusing optics for some large field confocal imaging apparatusesmaintain the same magnification with changes in focusing settings (e.g.,with changes in a position of one or more lenses along an imaging axis).However, embodiments of the present invention provide a fieldcompensator (see, e.g., field compensator 92 of FIG. 1B) that minimizesor eliminates the error introduced by the change in magnification.

FIGS. 3A-3B illustrate a probing member 300 in accordance with oneembodiment. The probing member 300 is made of a light transmissivematerial such as glass. In one embodiment, the probing member 300 actsas a prism and corresponds to prism 220 of FIG. 2 . Probing member 300may include an anterior segment 301 and a posterior segment 302, tightlybonded (e.g., glued) in an optically transmissive manner at 303. Probingmember 300 may additionally include a slanted face 304 covered by areflective mirror layer 305. A window 306 defining a sensing surface 307may be disposed at a bottom end of the anterior segment 301 in a mannerleaving an air gap 308. The window 306 may be fixed in position by aholding structure which is not shown. An array of light rays or beams309 are represented schematically. As can be seen, the array of lightbeams 309 are reflected at the walls of the probing member at an anglein which the walls are totally reflective and finally reflect on mirrorlayer 305 out through the sensing face 307. The array of light beams 309focus on a non-flat focal surface 310, the position of which can bechanged by the focusing optics (not shown in this figure).

Various components of the confocal imaging apparatus may dissipateconsiderable amounts of heat relative to a size of the confocal imagingapparatus. For example, the confocal imaging apparatus may include aCMOS sensor and an FPGA, both of which may produce heat. Accordingly,internal temperatures of the confocal imaging apparatus may rise overtime during use. At any given time, different portions of the confocalimaging apparatus may have different temperatures. A temperaturedistribution within the confocal imaging apparatus is referred to as athermal state of the confocal imaging apparatus. The thermal state ofthe confocal imaging apparatus may affect various optical parameters.For example, the thermal state may cause the positions of one or moreoptical components to move within the confocal imaging apparatus due toexpansion of the various components in accordance with thermal expansioncoefficients of these components. Additionally, the refractivecoefficient of one or more lens of the confocal imaging apparatus maychange with changes in the thermal state. Such changes causemeasurements produced by the confocal imaging apparatus to change withchanges in the internal thermal state. Some regions of the confocalimaging apparatus are more sensitive to thermal change than others(e.g., due to a high optical gain). For example, some optical elementsmay have an axial gain of up to about 7.5 in an embodiment. For suchoptical elements, a 10 μm movement due to changes in the thermal statecould cause up to a 75 μm shift in a measurement. Accordingly, in someembodiments, as shown in FIGS. 3C-3D, an internal target is used toadjust for measurement changes caused by changes in the thermal state.Alternatively, multiple temperature sensors may be disposed within theconfocal imaging apparatus and used to determine changes in the thermalstate.

FIGS. 3C-3D illustrate a probing member 370 that includes an internaltarget 380, in accordance with one embodiment. The probing member 370 issubstantially similar to probing member 300. For example, probing member370 may be made of a light transmissive material such as glass, and mayact as a prism. Probing member 370 may include an anterior segment 371and a posterior segment 372, tightly bonded (e.g., glued) in anoptically transmissive manner. Probing member 370 may additionallyinclude a slanted face covered by a reflective mirror layer. A window376 defining a sensing surface may be disposed at a bottom end of theanterior segment 371. The window 376 may be glass or another transparentmaterial, and may be fixed in position by a holding structure which isnot shown.

Probing member 370 additionally includes internal target 380 secured tothe anterior segment 371 of the probing member 370 within a field ofview (FOV) of the probing member 370. The internal target 380 may be arigid reflective material that will reflect light beams. The internaltarget 380 may be secured at a fixed position within the probing member300. Since the internal target 380 is a part of the probing member 370,the location of the internal target 380 should remain constant. In oneembodiment, the internal target 380 takes up approximately 500 μm to 1mm of the FOV.

During measurement, an array of light rays or beams 390-392 is projectedout of the anterior segment 371. As can be seen, the internal target 380is in the path of light beams 390. Accordingly, the light beams 390 arereflected off of the internal target 380, which provides a depth(z-axis) measurement of the internal target 380. Since the internaltarget 380 is at a fixed position, the measured depth of the internaltarget 380 should not change. Accordingly, any measured change in theposition of the internal target 380 reflects changes in internal opticsassociated with the thermal state of the confocal imaging apparatus.

The light beams 392 project through the window 376 and focus on anon-flat focal surface 310, the position of which can be changed by thefocusing optics (not shown in this figure). Alternatively, the internaltarget 380 may be included in an imaging apparatus with a flat focalsurface (e.g., an imaging apparatus with a field lens). Such an imagingapparatus may or may not be a confocal imaging apparatus. These lightbeams 392 may be used to measure the position of an object in the FOV ofthe confocal imaging apparatus. The measured change in the position ofthe internal target 380 can be used to correct for measurement errorscaused by the thermal state. Any apparent change in the z-axis positionof the internal target 380 may be used to apply an adjustment factor toother z-axis measurements of the imaged object to compensate for changesin the focusing optics caused by temperature. Additionally, a change inthe z-axis position of the internal target may be used to apply anadjustment to the X and Y pixel measurements in embodiments. In oneembodiment, the z-axis position of the internal target and measuredpoints of an object are input into a thermal state compensation model tocompensate for the thermal state. In one embodiment, the thermal statecompensation model is a three dimensional polynomial function.

FIG. 4 is a schematic illustration of a confocal imaging apparatus 450,in accordance with one embodiment. In one embodiment, the confocalimaging apparatus 450 corresponds to on focal imaging apparatus 20 ofFIG. 1A. In one embodiment, components of confocal imaging apparatus 20correspond to like named components illustrated in optics 200 of FIG. 2. In confocal imaging apparatus 450 a parent light beam 452 may be acombination of light emitted by multiple lasers 454A, 454B and 454C.Alternatively, the parent light beam 452 may be produced by a singlelaser (e.g., 454B). An illumination module 456 (e.g., an optic expander)then expands the single parent beam into an array of incident lightbeams 458. Incident light beams pass through a unidirectional (e.g.,unidirectional) mirror or beam splitter 460, then through focusingoptics 462 towards an object 464 to be imaged.

Parent beam 452 may include multiple different wavelengths, with adifferent wavelength being transmitted from each laser 454A-C. Thus,parent light beam 452 and one or more incident light beams in the arrayof light beams 458 may be composed of multiple different lightcomponents. Alternatively, each light beam in the array of light beamsmay include a single wavelength from the multiple wavelengths of parentbeam 452. Lasers 454A-C may be arranged such that each light beamfocuses on a different curved focal surface, P_(A), P_(B) and P_(C),respectively. In the position shown in FIG. 4 , incident light beam 458Areflects off of the surface at spot 470A, which in the specific opticalarrangement of optics 462 is in the focal point for light component A(emitted by laser 454A). Thus, a returned light beam 472A is measured bya detector 476 that includes a two dimensional array of sensors, eachcorresponding to a pixel. In one embodiment, the detector is atwo-dimensional array of spectrophotometers, e.g. a 3 CHIP CCD sensor.Similarly, different maximal intensity will be reached for spots 470Band 470C for light components B and C, respectively. Thus, by usingdifferent light components each one focused simultaneously at adifferent plane, the time used to complete a measurement can be reducedas different focal plane ranges can simultaneously be measured.

In an alternative embodiment, only a single wavelength of light isemitted (e.g., by a single laser). Thus, parent beam 452 and the arrayof light beams 458 may include a single wavelength. In such anembodiment, each of the light beams in the array of light beams 458focuses on the same curved focal surface P_(C). Thus in the positionshown in FIG. 4 , incident light beam 458A reflects off of the surfaceat spot 470A which in the specific focusing setting of focusing optics462 is at the focal point for focusing optics 462. Thus, the returnedlight beam 472A is measured by a detector 476 that includes a twodimensional array of sensors, each corresponding to a pixel and isregistered as the z-axis position for spot 470C. Similarly, incidentlight beams 458A, 458B reflect off of the surface at spots 470A and470B, respectively. However, the spots 470A, 470B are not on the curvedfocal surface P_(C). Accordingly, light is reflected back in a blurredmanner from the object 464 for those spots. By changing the focusingsetting for focusing optics 462 so that the focal point aligns with spot4708 and separately with 470A, corresponding depths associated withthose focusing settings may be detected for spots 470B and 470A,respectively.

FIG. 5A is a flow chart showing one embodiment of a method 500 forcalibrating a confocal imaging apparatus having an imaginary non-flatfocal surface. Method 500 may be performed by processing logic that maycomprise hardware (e.g., circuitry, dedicated logic, programmable logic,microcode, etc.), software (e.g., instructions run on a processingdevice to perform hardware simulation), or a combination thereof. In oneembodiment, at least some operations of method 500 are performed by acomputing device (e.g., computing device 24 of FIG. 1B). In oneembodiment, at least some operations of method 500 are performed byconfocal imaging apparatus 20 of FIG. 1A.

The confocal imaging apparatus described in embodiments herein has anon-flat (e.g., curved) focal surface. This curved focal surfaceintroduces inaccuracies in depth measurements of points of a scannedobject. For example, a first point of the object at a center of theconfocal imaging apparatus' imaging field may be in focus and thus causea highest intensity measurement at a depth Z_(i). However, a secondpoint of the object at an edge of the imaging field that has a samedepth as the first point may be in focus and cause a highest intensitymeasurement at a depth Z_(i)+X due to the non-flat focal surface, whereX represents the difference between the focal point at the center of theimaging field and the focal point at the edge of the imaging field.Thus, the non-flat imaging field will cause measurements of the firstand second points to yield different depth values even though they areat the same depth. In one embodiment, calibration method 500 isperformed to calibrate the confocal imaging apparatus so that the errorintroduced by the non-flat focal surface can be eliminated.

At block 505 of method 500, a calibration object is measured by theconfocal imaging apparatus. The calibration object is a high accuracyobject with known X, Y and Z coordinates for every point of thecalibration object. The accuracy level of the calibration object maydefine the final accuracy of the confocal imaging apparatus. In oneembodiment, the X, Y and Z coordinates for the calibration object areaccurate and known to a level of accuracy that is a degree of magnitudehigher than a final desired accuracy of the confocal imaging apparatus.For example, if the confocal imaging apparatus is to have a finalaccuracy to 5 microns, then the calibration object may be accurate to0.5 microns.

Various calibration objects may be used, a few examples of which are setforth herein. One example calibration object is a sphere with a veryaccurate radius on an accurate X-Y-Z stage. Another example calibrationobject is a flat plate with a grid of horizontal and vertical linesprinted on a surface of the plate. A flatness of the plate and the linespacing may be very accurate. Another example calibration object is aflat plate with circles or dots printed on a surface of the plate. Theflatness of the plate and the size and spacing of the circles may bevery accurate. Many other calibration objects may also be used. FIG. 5Cillustrates one example calibration object 590, which is a flat platewith a grid of precisely spaced circles or dots.

Referring back to FIG. 5A, the calibration object is measured at eachfocusing setting (e.g., encoder position) of the confocal imagingapparatus. For some types of calibration objects (e.g., the sphere), thecalibration object is moved to multiple different X, Y positions foreach focusing setting and/or to multiple different X, Y, Z positions foreach focusing setting. For other types of calibration objects (e.g., theplates), the calibration object may be moved to multiple different Zpositions for each focusing setting. Measurements may be taken for eachposition of the calibration object.

In one embodiment, the calibration object is mounted to a calibrationjig, which may precisely move the calibration object in one or moredimensions. For example, the calibration object 590 may be mounted tothe calibration jig, and the calibration jig may be moved along thez-axis. In one embodiment, the calibration jig moves the calibrationobject in 1 mm increments, with an accuracy of 1 μm. The calibration jigmay move the calibration object in such a way as to cover more than thefull field of view of the confocal imaging apparatus (e.g., thecalibration object may be larger than the FOV of the confocal imagingapparatus) and to cover more than the range for the depth of scanning ofthe confocal imaging apparatus.

In the example of the calibration object 590, the calibration object 590may be scanned in two ways. A first scan may be performed at each depthposition of the calibration object 590 using regular confocal scanning.This will provide a z-position for each dot in the coordinate system ofthe confocal imaging apparatus (e.g., based on the coordinates of theencoder that positions the lens). A second scan may be performed togenerate an image of the dots at focus for each focal setting. The imagemay be used to determine an X, Y position for the center of each dot inpixel coordinates and with sub-pixel accuracy.

At block 510, the measurements of the calibration object (measurementsof the calibration object's surface topology) are compared to a knownsurface topology of the calibration object. Each point in thecalibration object (e.g., each dot in calibration object 590 having ameasured x-pixel, y-pixel and encoder value) may be paired to acorresponding real world point (point in a world coordinate system) fromthe calibration object, where the world coordinate system corresponds toknown X, Y, Z coordinates of the calibration object. For example, the Xand Y coordinates for calibration object 590 would correspond to knownfixed positions of the dots, and the Z coordinate for calibration object590 would depend on a setting of a calibration jig. For each point ofthe calibration object, a difference between a measured depth value anda known depth value may be determined. Additionally, for each point ofthe calibration object, a difference between a measured X and Y positionand a known X and Y position may be determined. This may be performedfor each focusing setting of the confocal imaging apparatus.

At block 515, the determined differences of the multiple points may beapplied to a smooth function (e.g., to a polynomial function such as athree dimensional polynomial function) that may be used to model thefield curvature of the confocal imaging apparatus' non-flat focalsurface. The function is referred to herein as a un-distortion function.In one embodiment, the determined differences are applied to solve forthe constants in a bivariate quadratic polynomial of the form:

Z _(Field Curvature (object))(x,y,Z _(optics))=a ₁ x ² +a ₂ y ² +a ₃ x+a₄ y+a ₅ xy+a ₆  (1)

Where x and y are the X, Y coordinates for points on a plane normal tothe imaging axis. Alternatively, a higher order polynomial may be used.The smooth function with the solved constants may then be used as anaccurate field curvature model. Every parameter may be a polynomial thatdepends on the focusing setting (z-axis value) of the confocal imagingapparatus. This may result in an 18 parameter field curvature model ifthe above described bivariate quadratic polynomial is used.

Alternatively, the determined differences may be applied to solve forthe constants in another smooth function (e.g., a function describing aconic shape). In such an embodiment, a generated model may have adifferent number of parameters (e.g., 12 parameters if a functiondescribing a conic shape is used). Linear minimization methods (e.g.,linear least square method) and/or non-linear minimization methods(e.g., Broyden-Fletcher-Goldfarb-Shanno (BFGS) method) may be applied tofind the best values for the constants. As mentioned, this process maybe performed for each focusing setting. This is because the amount offield curvature may change with different focusing settings of theconfocal imaging apparatus. Accordingly, a separate field curvaturemodel may be generated for each focusing setting. Alternatively, asingle field curvature model may be generated that accounts for thechanges to the field curvature model due to changes in the focusingsetting.

In embodiments, X and Y positions are solved for at the same time thatthe depth is solved for. For example, differences in X and Y position atdifferent focus settings may also be applied to solve for the constantsin the smooth function. Additionally, other types of geometriccorrection may be solved for as well using this technique. All suchgeometric corrections may be solved for together. Other types ofphenomena that may be corrected for using this technique includemagnification change, optical distortion (e.g., non-constantmagnification in x and y), optical aberrations, and so on. All suchdistortions may be solved for together.

FIG. 5D illustrates a chart 594 showing a distribution of points of thecalibration object 590 as measured by the confocal imaging apparatus (inthe coordinate system of the confocal imaging apparatus). Chart 594shows measurements taken with the calibration object 590 at threedifferent z positions. As shown, the dots appear to lie on a curvedsurface. FIG. 5E illustrates a chart 597 showing a distribution ofpoints in the real world. Chart 597 shows measurements taken with thecalibration object 590 at three different z positions. As shown, thedots lie on a plane. After calibration, the transformation for each dotmay be determined to correct for optical distortions. Thus, the trueworld position of each dot may be accurately measured.

At block 525, a temperature dependence of the confocal imaging apparatus(e.g., the focusing optics and of a lens housing for the focusingoptics) is determined. In one embodiment, the operations of one or moreof blocks 505-515 are performed at multiple temperatures over atemperature operating range of the confocal imaging apparatus todetermine the temperature dependence. Changes in temperature may causedifferences in the measured depth values. Accordingly, a temperaturedependency may be determined and applied to the field curvature model tocreate a thermal state correction model. For example, the fieldcurvature model may be modified from x, y, z=F(i, j, encoder) to x, y,z=F(i, j, encoder, T_(state)), where x, y and z represent real worldcoordinates, i represents an x-pixel, j represents a y-pixel, encoderrepresents a focal setting (encoder position), and T_(state) representsa thermal state. For such a model that takes into account the thermalstate, an estimate of the thermal state should be obtained for eachmeasurement. A thermal state correction model may also be generated foran imaging apparatus with a flat focal surface using the same process asdescribed herein for an imaging apparatus with a curved focal surface.

In one embodiment, opto-mechanical simulation is performed to determinea relationship between temperature and adjustments in calibration of thefocusing optics. This relationship may be used to determine a correctionthat may be applied to all parameters of the generated field curvaturemodel or models, where the amount of correction is based on a currenttemperature.

In one embodiment, the main change in the focusing optics due totemperature is a focus shift. Curvature of the non-flat focal surfacemay be practically unchanged by changes in temperature. In oneembodiment, a shift in focus for focusing settings may be determined byscanning one or more elements (e.g., an internal target such as internaltarget 380 of FIGS. 3C-3D) of the confocal imaging apparatus that isnear or along the optical path. In one embodiment, the scanned elementis on a side of a field of view (FOV) of the confocal imaging apparatus.This element may be kept at the same distance relative to one or morecomponents of the focusing optics. With each scan, when the 3D surfaceof an object is captured, the edge of the FOV where the internal targetis located captures a position of the internal target. Due to the factthat the internal target is part of the confocal imaging apparatus andhas a fixed position, detected changes in the position of the internaltarget are caused by changes in the thermal state. Accordingly, if afocus shift of the internal target is detected from the scan, then anadjustment factor may be applied to the field curvature model tocompensate for the thermal state.

In one embodiment, separate field curvature models are generated foreach temperature value or range of the confocal imaging apparatus at aparticular focusing setting. Alternatively, a single model may begenerated for each focusing setting that accounts for changes intemperature. Alternatively, a temperature dependent adjustment factormay be determined and applied to the field curvature model or modelsbased on a measured temperature.

In one embodiment, a simple model may be used that assumes that opticalchange caused by the thermal state is primarily due to a linear shift inthe focal setting (e.g., a backward motion in the encoder position). Forsuch a model, changes caused by the thermal state may be corrected byadding the difference between a current measured internal targetposition and a reference value to every focal setting (encoder value)before applying the un-distortion function. The simple model may havethe form of:

x,y,z=F(i,j,encoder−(internal target position−reference targetposition))  (2)

where F is the un-distortion function, such as function (1) above.

In another embodiment, a more complex model is used that assumesinternal target effects are caused by the focal shift of encoder, but ina complex way. Such a model may have the form of:

x,y,z=F(i,j,f(encoder,internal target position))  (3)

In another embodiment, a model that corrects for distortions caused bythe thermal state assumes that the thermal state changes all optics by asmall amount that can be linearly estimated. Such a model may have theform of:

$\begin{matrix}{x,y,{z = {{{F_{hot}\left( {i,j,{encoder}} \right)}\frac{\left( {p - a} \right)}{\left( {b - a} \right)}} + {{F_{cold}\left( {i,j,{encoder}} \right)}\left( {1 - \frac{\left( {p - a} \right)}{\left( {b - a} \right)}} \right)}}}} & (4)\end{matrix}$

where F_(hot) is the un-distortion function under a hot condition,F_(cold) is the un-distortion function under a cold condition, a is theinternal target position in the hot condition, b is the internal targetposition in the cold position, and p is the measured internal targetposition.

At block 535, the one or more generated field curvature models for theconfocal imaging apparatus are stored. The field curvature models may bestored in a memory of the confocal imaging apparatus and/or in a memoryof a computing device that processes data from the confocal imagingapparatus. In one embodiment, the field curvature models are stored in anonvolatile memory (e.g., a read only memory (ROM), FLASH, or othernonvolatile memory) of the confocal imaging apparatus. The Fieldcurvature model (or models) may be applied to measurements of theconfocal imaging apparatus to correct the error in the depthmeasurements that are introduced by the non-flat focal surface of theconfocal imaging apparatus. If calibration information is stored inmemory of the confocal imaging apparatus, then the field curvaturemodels may be sent along with measurement data to a computing devicewhen measurements are taken. The computing device may then use thereceived field curvature models to correct for the field curvature ofthe confocal imaging apparatus.

FIG. 5B is a flow chart showing one embodiment of a method 550 forcalibrating a confocal imaging apparatus for which changes in a focusingsetting cause changes in magnification. Method 550 may be performed byprocessing logic that may comprise hardware (e.g., circuitry, dedicatedlogic, programmable logic, microcode, etc.), software (e.g.,instructions run on a processing device to perform hardware simulation),or a combination thereof. In one embodiment, at least some operations ofmethod 550 are performed by a computing device (e.g., computing device24 of FIG. 1B). In one embodiment, at least some operations of method550 are performed by confocal imaging apparatus 20 of FIG. 1A.

The confocal imaging apparatus described with reference to method 550may have a non-flat (e.g., curved) focal surface or a flat focal plane.Moreover, the confocal imaging apparatus described with reference tomethod 550 has focusing optics that are configured so that changes in afocusing setting cause a change in magnification at the focal surface orfocal plane. This change in magnification introduces inaccuracies in Xand Y position measurements of points of a scanned object. For example,a point of the object might be measured to have a first X and Y positionat a first focusing setting, but might be measured to have a second Xand Y position at a second focusing setting. Thus, the magnificationchanges will cause measurements to yield different X, Y values as thefocusing setting changes. In one embodiment, calibration method 550 isperformed to calibrate the confocal imaging apparatus so that theinaccuracies introduced by the changes in magnification can beeliminated.

At block 555 of method 500, a calibration object is measured by theconfocal imaging apparatus. The calibration object is a high accuracyobject with known X, Y and Z coordinates for every point of thecalibration object. The accuracy level of the calibration object maydefine the final accuracy of the confocal imaging apparatus. In oneembodiment, the X, Y and Z coordinates for the calibration object areaccurate and known to a level of accuracy that is a degree of magnitudehigher than a final desired accuracy of the confocal imaging apparatus.For example, if the confocal imaging apparatus is to have a finalaccuracy to 5 microns, then the calibration object may be accurate to0.5 microns. Any of the calibration objects described with reference toFIG. 5A may be used.

The calibration object is measured at each focusing setting (encodervalue) of the confocal imaging apparatus. For some types of calibrationobjects (e.g., the sphere), the calibration object is moved to multipledifferent X, Y positions for each focusing setting and/or to multipledifferent X, Y, Z positions for each focusing setting. For other typesof calibration objects (e.g., the plates), the calibration object may bemoved to multiple different Z positions for each focusing setting.Measurements may be taken for each position of the calibration object.Based on these measurements, a list of coordinates is collected in boththe calibration object space (e.g., real world) and in the sensor/opticsspace (e.g., virtual space). In the calibration object space, each setof coordinates for a point of the object has an X_(obj), Y_(obj) andZ_(obj) coordinate. These coordinates are known to be accurate due tothe known information about the calibration object. In the sensor/opticsspace, each set of coordinates for a point of the object includes anX_(pix), Y_(pix), Z_(optics) coordinate, where X_(pix) and Y_(pix) aredetermined based on the pixel detecting the point and Z_(optics) is thelens position of the focusing optics (e.g., the focusing setting).

At block 560, the measurements of the calibration object (measurementsof the calibration object's surface topology) may be compared to a knownsurface topology of the calibration object. For each point of thecalibration object, a difference between a measured depth value, X valueand/or Y value and a known depth value, X value and/or Y value may bedetermined. This may be performed for each focusing setting of theconfocal imaging apparatus.

At block 562, it is determined whether the focusing optics have a curvedfocal surface. If the focusing optics do have a curved focal surface,the method proceeds to block 565. Otherwise the method proceeds to block570.

At block 565, the determined differences of the multiple points for theX, Y and/or Z coordinates may be applied to a smooth function (e.g., toa polynomial function such as a three dimensional polynomial function)that may be used to model the field curvature of the confocal imagingapparatus' non-flat focal surface. In one embodiment, the determineddifferences are applied to solve for the constants in a bivariatequadratic polynomial of the form:

Z _(Field Curvature (object))(x,y,Z _(optics))=a ₁ x ² +a ₂ y ² +a ₃ x+a₄ y+a ₅ xy+a ₆  (5)

Where x and y are the X_(pix), Y_(pix) coordinates in the sensor space.Alternatively, the determined differences may be applied to solve forthe constants in another smooth function (e.g., a function describing aconic shape), such as a polynomial of higher order. The smooth functionwith the solved constants may then be used for an accurate fieldcurvature model.

At block 570, the determined differences of the multiple points for theX, Y and/or Z coordinates may be applied to a smooth function (e.g., toa polynomial function such as a three dimensional or higher dimensionalpolynomial function) that may be used to model the changes inmagnification of the confocal imaging apparatus on an x-axis caused bychanges in the focusing setting (e.g., changes in the Z_(optics) value).In one embodiment, the determined differences are applied to solve forthe constants in a bivariate quadratic polynomial of the form:

X _(Object)(x,y,Z _(optics))=b ₁ x ² +b ₂ y ² +b ₃ x+b ₄ y+b ₅ xy+b₆  (6)

Where x and y are the X_(pix), Y_(pix) coordinates in the sensor space.Alternatively, the determined differences may be applied to solve forthe constants in another smooth function (e.g., in another threedimensional polynomial function, such as a function describing a conicshape). The smooth function with the solved constants may then be usedas an accurate magnification compensation model for the X coordinate.

At block 575, the determined differences of the multiple points for theX, Y and/or Z coordinates may be applied to a smooth function (e.g., toa polynomial function such as a three dimensional polynomial function)that may be used to model the changes in magnification of the confocalimaging apparatus on a y-axis caused by changes in the focusing setting(e.g., changes in the Z_(optics) value). In one embodiment, thedetermined differences are applied to solve for the constants in abivariate quadratic polynomial of the form:

Y _(Object)(x,y,Z _(optics))=c ₁ x ² +c ₂ y ² +c ₃ x+c ₄ y+c ₅ xy+c₆  (7)

Where x and y are the X_(pix), Y_(pix) coordinates in the sensor space.Alternatively, the determined differences may be applied to solve forthe constants in another smooth function (e.g., in another threedimensional polynomial function, such as a function describing a conicshape). The smooth function with the solved constants may then be usedas an accurate magnification compensation model for the Y coordinate.

Blocks 565, 570 and 575 have been described as three separateoperations. However, in some embodiments a single operation may beperformed to solve for each of the x-coordinate, the y-coordinate andthe z-coordinate. For example, an un-distortion function having thefollowing form may be solved to determine the x, y and z coordinates.

F _(X)(x,y,z)=a ₀ +a ₁ x+a ₂ y+a ₃ z+a ₄ x ² +a ₅ y ² +a ₆ z ² + . . .+a _(i) xy+ . . . +a _(j) x ^(n) y ^(m) z ^(k)

F _(Y)(x,y,z)=b ₀ +b ₁ x+b ₂ y+b ₃ z+b ₄ x ² +b ₅ y ² +b ₆ z ² + . . .+b _(i) xy+ . . . +b _(j) x ^(n) y ^(m) z ^(k)

F _(Z)(x,y,z)=c ₀ +c ₁ x+c ₂ y+c ₃ z+c ₄ x ² +c ₅ y ² +c ₆ z ² + . . .+c _(i) xy+ . . . +c _(j) x ^(n) y ^(m) z ^(k)  (8)

where F_(X), F_(Y) and F_(Z) are the functions whose results in worldcoordinates are to be solved for, x and y are pixel coordinates measuredby the confocal imaging apparatus, z is a focal setting (e.g., encodercoordinates corresponding to a focal setting), a_(i), b_(i) and c_(i)are learned parameters, and n, m and k are the maximal degree of thenominal. The function may be selected to minimize a mean square errorbetween the world coordinates and the found positions after the functiontransformation. Outlier positions may be detected and removed beforefitting. In one embodiment, a number of non-zero parameters isconstrained.

At block 580, one or more optics correcting models are generated basedon the first second and third polynomial functions (or other smoothfunctions), such as those represented in equations 5-8. Every parameterfor equations 5-8 may be a polynomial that depends on the focusingsetting (z-axis value) of the confocal imaging apparatus. In oneembodiment, each parameter is modeled as a quadratic change to theZ_(optics) (focusing setting). For example, parameter a may be aparameter having a form:

a ₁(Z _(Optics))=A+B*Z _(Optics) +C*Z _(Optics) ²  (9)

Parameters a₂-a₆, b₁-b₆ and c₁-c₆ may be similarly represented. This mayresult in a 54 parameter model that corrects for full curvature,magnification and distortion of the field of view (FOV).

Linear minimization methods (e.g., linear least square method) and/ornon-linear minimization methods (e.g., Broyden-Fletcher-Goldfarb-Shanno(BFGS) method) may be applied to find the best values for the constantsat each of blocks 565, 570 and 575. As mentioned, these processes may beperformed for each focusing setting. This is because the amount of fieldcurvature and magnification may change with different focusing settingsof the confocal imaging apparatus. Accordingly, a separate model may begenerated for each focusing setting. Alternatively, a single model maybe generated that accounts for the changes to the model due to changesin the focusing setting. Note that temperature dependence may also bedetermined and included in the model as described with reference toblock 525 of method 500. In one embodiment, a temperature dependence isdetermined, and a model that corrects for thermal state is created, asdiscussed above with reference to method 500.

At block 585, the one or more generated models for the confocal imagingapparatus are stored. The models may be stored in a memory of theconfocal imaging apparatus and/or in a memory of a computing device thatprocesses data from the confocal imaging apparatus. In one embodiment,the models are stored in a nonvolatile memory (e.g., a read only memory(ROM), FLASH, or other nonvolatile memory) of the confocal imagingapparatus. The model (or models) may be applied to measurements of theconfocal imaging apparatus to correct the error in the depthmeasurements that are introduced by the non-flat focal surface as wellas to correct for inaccuracies caused by changes in magnification. Ifcalibration information is stored in memory of the confocal imagingapparatus, then the models may be sent along with measurement data to acomputing device when measurements are taken. The computing device maythen use the received models to correct for the field curvature and/ormagnification changes of the confocal imaging apparatus.

FIG. 6 is a flow chart showing one embodiment of a method 600 foradjusting depth measurements of a scanned three dimensional object basedon application of a field curvature model or other model (e.g., athermal state compensation model) calibrated to a confocal imagingapparatus or other imaging apparatus (e.g., a stereoscopic imagingapparatus). Method 600 may be performed by processing logic that maycomprise hardware (e.g., circuitry, dedicated logic, programmable logic,microcode, etc.), software (e.g., instructions run on a processingdevice to perform hardware simulation), or a combination thereof. In oneembodiment, at least some operations of method 600 are performed by acomputing device (e.g., computing device 24 of FIG. 1B executing imageprocessing module 82).

At block 605 of method 600, processing logic receives intensitymeasurements generated by pixels of a detector of a confocal imagingapparatus. The detector may have a two-dimensional array of pixels, andeach pixel may receive a particular light beam of an array of lightbeams directed at the detector. The array of light beams may be an arrayof returning light beams that have been reflected off of a surface ofthe imaged three dimensional object. Thus, each pixel of the detector isassociated with a particular point of the three dimensional object andprovides intensity measurements for an associated returning light beamfrom the array of returning light beams.

Each received intensity measurement is associated with a particularfocusing setting of the confocal imaging apparatus. Intensitymeasurements may be received over a range of focusing settings. At block620, processing logic determines, for each pixel, a focusing setting ofthe confocal imaging apparatus that provides a maximum measuredintensity.

A relative distance between a probe of the confocal imaging apparatusand a focal point of the confocal imaging apparatus may be known foreach focusing setting (encoder value). A point of the imaged object isknown to be in focus (e.g., at the focal point) when a measuredintensity for that point is maximal. Accordingly, at block 630processing logic determines, for each pixel, a depth of a point of thethree dimensional object associated with that pixel that corresponds tothe focusing setting that yielded the maximal intensity. If the imagingapparatus includes an internal target in the FOV of the imagingapparatus, then some pixels will be associated with points on theinternal target. Accordingly, a depth of the points of the internaltarget may also be determined.

As discussed previously herein, the non-flat focal surface and/ormagnification changes of the confocal imaging apparatus introduce anerror in the depth measurements and/or in the X, Y coordinatemeasurements. Accordingly, at block 640 processing logic adjusts thedetermined depths of points of the imaged three dimensional object basedon applying the determined focusing settings for the pixels associatedwith those points to a field curvature model. Processing logic mayadditionally or alternatively determine X, Y coordinates of the pointsbased on applying the determined focusing settings to the fieldcurvature model or other model. One or more field curvature modelsand/or other models may be used. For example, a particular fieldcurvature model and/or other model may be associated with each focusingsetting. An appropriate field curvature model may be identified based onthe focusing setting at which a point on the object came into focus. Aparticular depth adjustment for that point may then be determined byproviding the X, Y coordinates of the pixel into the determined fieldcurvature model. Alternatively, a single field curvature model may beused, and the X, Y coordinates and focusing setting may be input intothe field curvature model to determine the depth displacement. In oneembodiment, a temperature of the focusing optics is also measured and/ora thermal state is otherwise determined (e.g., using an internal targetposition), and an additional depth adjustment factor (and/or otheroptical adjustment) is determined based on the temperature (e.g., usinga thermal state compensation model). This additional depth adjustmentfactor (and/or additional optical adjustment) may then be applied to themeasured depths (and/or X and Y coordinates) of all points. In oneembodiment, a single model is used that compensates for both the thermalstate and field curvature.

At block 650, processing logic may determine a shape (e.g., surfacetopology) of the three dimensional object based on the adjusted depthsand/or x and y coordinates. Processing logic may then create an accuratevirtual three dimensional model of the imaged object.

FIG. 7 illustrates a diagrammatic representation of a machine in theexample form of a computing device 700 within which a set ofinstructions, for causing the machine to perform any one or more of themethodologies discussed herein, may be executed. In alternativeembodiments, the machine may be connected (e.g., networked) to othermachines in a Local Area Network (LAN), an intranet, an extranet, or theInternet. The machine may operate in the capacity of a server or aclient machine in a client-server network environment, or as a peermachine in a peer-to-peer (or distributed) network environment. Themachine may be a personal computer (PC), a tablet computer, a set-topbox (STB), a Personal Digital Assistant (PDA), a cellular telephone, aweb appliance, a server, a network router, switch or bridge, or anymachine capable of executing a set of instructions (sequential orotherwise) that specify actions to be taken by that machine. Further,while only a single machine is illustrated, the term ‘machine’ shallalso be taken to include any collection of machines (e.g., computers)that individually or jointly execute a set (or multiple sets) ofinstructions to perform any one or more of the methodologies discussedherein. In one embodiment, computing device 700 corresponds to computingdevice 24 of FIG. 1B.

The example computing device 700 includes a processing device 702, amain memory 704 (e.g., read-only memory (ROM), flash memory, dynamicrandom access memory (DRAM) such as synchronous DRAM (SDRAM), etc.), astatic memory 706 (e.g., flash memory, static random access memory(SRAM), etc.), and a secondary memory (e.g., a data storage device 728),which communicate with each other via a bus 708.

Processing device 702 represents one or more general-purpose processorssuch as a microprocessor, central processing unit, or the like. Moreparticularly, the processing device 702 may be a complex instruction setcomputing (CISC) microprocessor, reduced instruction set computing(RISC) microprocessor, very long instruction word (VLIW) microprocessor,processor implementing other instruction sets, or processorsimplementing a combination of instruction sets. Processing device 702may also be one or more special-purpose processing devices such as anapplication specific integrated circuit (ASIC), a field programmablegate array (FPGA), a digital signal processor (DSP), network processor,or the like. Processing device 702 is configured to execute theprocessing logic (instructions 726) for performing operations and stepsdiscussed herein.

The computing device 700 may further include a network interface device722 for communicating with a network 764 or other device. The computingdevice 700 also may include a video display unit 710 (e.g., a liquidcrystal display (LCD) or a cathode ray tube (CRT)), an alphanumericinput device 712 (e.g., a keyboard), a cursor control device 714 (e.g.,a mouse), and a signal generation device 720 (e.g., a speaker).

The data storage device 728 may include a machine-readable storagemedium (or more specifically a non-transitory computer-readable storagemedium) 724 on which is stored one or more sets of instructions 726embodying any one or more of the methodologies or functions describedherein. A non-transitory storage medium refers to a storage medium otherthan a carrier wave. The instructions 726 may also reside, completely orat least partially, within the main memory 704 and/or within theprocessing device 702 during execution thereof by the computer device700, the main memory 704 and the processing device 702 also constitutingcomputer-readable storage media.

The computer-readable storage medium 724 may also be used to store afield compensator 750 which may correspond to field compensator 92 ofFIG. 1B. The computer readable storage medium 724 may also store asoftware library containing methods that call the field compensator 750.While the computer-readable storage medium 724 is shown in an exampleembodiment to be a single medium, the term “computer-readable storagemedium” should be taken to include a single medium or multiple media(e.g., a centralized or distributed database, and/or associated cachesand servers) that store the one or more sets of instructions. The term“computer-readable storage medium” shall also be taken to include anymedium that is capable of storing or encoding a set of instructions forexecution by the machine and that cause the machine to perform any oneor more of the methodologies of the present invention. The term“computer-readable storage medium” shall accordingly be taken toinclude, but not be limited to, solid-state memories, and optical andmagnetic media.

It is to be understood that the above description is intended to beillustrative, and not restrictive. Many other embodiments will beapparent upon reading and understanding the above description. Althoughembodiments of the present invention have been described with referenceto specific example embodiments, it will be recognized that theinvention is not limited to the embodiments described, but can bepracticed with modification and alteration within the spirit and scopeof the appended claims. Accordingly, the specification and drawings areto be regarded in an illustrative sense rather than a restrictive sense.The scope of the invention should, therefore, be determined withreference to the appended claims, along with the full scope ofequivalents to which such claims are entitled.

What is claimed is:
 1. A non-calibrated intraoral scanning system to bemodified to a calibrated intraoral scanning system based on generating athree-dimensional (3D) representation of a calibration object,comprising: An intraoral scanning device, comprising: alight sourceconfigured to emit light, wherein the intraoral scanning device is totransmit the light onto the calibration object; and an image sensorcomprising a plurality of sensor elements, wherein the image sensor isto form a sequence of images of the calibration object, each imagecomprising a plurality of pixels; and a processor to perform acalibration of the non-calibrated intraoral scanning system, wherein theprocessor is to: import a reference 3D representation of the calibrationobject in a real-world coordinate system; transform, using one or moretransformations, at least a set of the pixels from each image of thesequence of images to a dataset of coordinates of the calibration objectin a real-world coordinate system; align the dataset of coordinates ofthe calibration object in the real-world coordinate system to thereference 3D representation of the calibration object in the real-worldcoordinate system; derive a difference between the dataset ofcoordinates of the calibration object and the reference 3Drepresentation of the calibration object in the real-world coordinatesystem; and modify, based on the difference, at least one of the one ormore transformations such that a location of one or more points in thedataset of coordinates of the calibration object in the real-worldcoordinate system is adjusted in three dimensions, wherein thenon-calibrated intraoral scanning system is modified to a calibratedintraoral scanning system.
 2. The non-calibrated intraoral scanningsystem according to claim 1, wherein the intraoral scanning device is ahandheld scanning device.
 3. The non-calibrated intraoral scanningsystem according to claim 1, wherein the light source comprises asemiconductor laser.
 4. The non-calibrated intraoral scanning systemaccording to claim 1, wherein: the image sensor is located in a planenormal to a z-direction in a intraoral scanning device coordinatesystem, and an optical element is configured to move relative to theimage sensor and along the z-direction, wherein the sequence of imagesof the calibration object is formed while the optical element is moved.5. The non-calibrated intraoral scanning system according to claim 4,wherein the processor is further to: compute a focus-measure for one ormore pixels of the plurality of pixels in the sequence of images; andcompute, based on the focus-measure, a maximum focus measure, Z_(m), forthe one or more pixels, wherein a subset of the one or more pixels getsassociated with Z_(m).
 6. The non-calibrated intraoral scanning systemaccording to claim 1, wherein modifying the one or more transformationsis performed by individually adjusting one or more of the points in thedataset of coordinates of the calibration object.
 7. The non-calibratedintraoral scanning system according to claim 1, wherein modifying theone or more transformations is performed by adjusting a 3D location inthree directions for one or more of the points in the dataset ofcoordinates of the calibration object.
 8. The non-calibrated intraoralscanning system according to claim 1, wherein modifying the one or moretransformations is performed by adding a correction vector, comprisingthree direction components, to one or more of the points in the datasetof coordinates of the calibration object.
 9. The non-calibratedintraoral scanning system according to claim 1, wherein modifying theone or more transformations is performed in association with a positionalong a z-direction of an optical element of the intraoral scanningdevice.
 10. The non-calibrated intraoral scanning system according toclaim 9, wherein modifying the one or more transformation comprisesapplying an adjustment in three dimensions for one or more points in thedataset, as associated with the position along the z-direction of theoptical element, the adjustment being defined by a higher orderpolynomial for the position along the z-direction.
 11. Thenon-calibrated intraoral scanning system according to claim 1, whereinthe calibration object is a two-dimensional (2D) calibration object. 12.The non-calibrated intraoral scanning system according to claim 1,wherein importing the reference 3D representation of the calibrationobject in the real-world coordinate system is based on importing a highprecision 3D representation of the calibration object.
 13. Thenon-calibrated intraoral scanning system according to claim 1, furthercomprising: the calibration object.
 14. The non-calibrated intraoralscanning system according to claim 13, wherein the calibration object isa 3D calibration object that comprises a known x value, a known y valueand a known z value for each of a plurality of points to within 0.5microns of accuracy.
 15. A computer-implemented method for calibratingan intraoral scanner, comprising: obtaining reference data of areference three-dimensional (3D) representation of a calibration object;obtaining, based on the intraoral scanner being used by a user to scanthe 3D calibration object, and from one or more device to real-worldcoordinate transformations of two-dimensional (2D) images of the 3Dcalibration object, measurement data; aligning the measurement data tothe reference data to obtain alignment data; and updating, based on thealignment data, said one or more transformations, to calibrate theintraoral scanner.
 16. The computer-implemented method according toclaim 15, wherein the updating is associated with a compensation model.17. The computer-implemented method according to claim 16, wherein amagnification of a focal surface of the intraoral scanner changes with afocusing setting of the intraoral scanner, and wherein the compensationmodel compensates for measurement errors of the intraoral scanner causedby changes in the magnification of the focal surface.
 18. Thecomputer-implemented method according to claim 16, wherein thecompensation model compensates for at least one of optical distortionsor optical aberrations of the intraoral scanner.
 19. Thecomputer-implemented method according to claim 15, wherein updating theone or more transformations comprises applying an adjustment in threedimensions for one or more points of the measurement data, as associatedwith a position along a z-direction of an optical element of theintraoral scanner.
 20. The computer-implemented method according toclaim 19, wherein the adjustment is defined by a higher order polynomialfor the position along the z-direction.
 21. The computer-implementedmethod according to claim 15, further comprising: determining one ormore differences between the measurement data and the reference data,wherein the updating is performed based on the one or more differences.